Automatic Complexity Determination of Gaussian Mixture Models with the EMS Algorithm

Estimating the complexity and regularisation parameters of semiparametric models like neural networks by repeated trials is slow, and makes them less attractive in real-time estimation problems. Simultaneous estimation of both model parameters and complexity can be achieved using the EMS algorithm which augments expectation-maximisation (EM) to include a pruning and growing step that relies on approximating the posterior odds of model structures with di erent complexities. EMS is applied to Gaussian mixtures. Regularising priors are introduced, including the truncated inverse exponential (TIE) distribution for the component covariance matrices. A fast method for estimating the hyperparameters tunes the smoothing action of the priors to the data. This approach is applied to density estimation of speech sound data and gives a signi cant performance advantage in comparison to current methods in speech recognition.